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Abstract  Results  obtained  from  self-reported  health  data 
may  be  biased  if  those  being  surveyed  respond  differently 
based  on  health  status.  This  study  was  conducted  to 
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investigate  if  health,  as  measured  by  health  care  use  pre¬ 
ceding  invitation,  influenced  response  to  invitation  to  a  21- 
year  prospective  study,  the  Millennium  Cohort  Study. 
Inpatient  and  outpatient  diagnoses  were  identified  among 
more  than  68,000  people  during  a  one-year  period  prior  to 
invitation  to  enroll.  Multivariable  logistic  regression 
defined  how  diagnoses  were  associated  with  response. 
Days  spent  hospitalized  or  in  outpatient  care  were  also 
compared  between  responders  and  nonresponders.  Adjus¬ 
ted  odds  of  response  to  the  questionnaire  were  similar  over 
a  diverse  range  of  inpatient  and  outpatient  diagnostic  cat¬ 
egories  during  the  year  prior  to  enrollment.  The  number  of 
days  hospitalized  or  accessing  outpatient  care  was  very 
similar  between  responders  and  nonresponders.  Study 
findings  demonstrate  that,  although  there  are  some  small 
differences  between  responders  and  nonresponders,  prior 
health  care  use  did  not  affect  response  to  the  Millennium 
Cohort  Study,  and  it  is  unlikely  that  future  study  findings 
will  be  biased  by  differential  response  due  to  health  status 
prior  to  enrollment  invitation. 

Keywords  Cohort  studies  •  Military  medicine  • 

Military  personnel  •  Response  bias  •  Veterans 

Abbreviations 

Cl  Confidence  interval 

ICD-9-  International  Classification  of  Diseases,  Ninth 

CM  Revision,  Clinical  Modification 

OR  Odds  ratio 

Introduction 

The  decision  to  participate  in  epidemiologic  health  studies 
is  multifaceted,  and  information  on  nonresponse  is  often 
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lacking  for  a  variety  of  reasons.  It  has  been  well  docu¬ 
mented  that  certain  groups  may  be  less  likely  to  respond  to 
surveys,  including  men,  young  adults,  those  of  lower 
socioeconomic  status,  and  unmarried  individuals  [1-4]. 
Basic  survey  research  asserts  that  the  prime  motivation  to 
respond  is  based  on  the  perceived  salience  of  the  study  to 
the  subject.  Heberlein  and  Baumgartner  conducted  a  sta¬ 
tistical  analysis  of  over  90  methodological  articles 
published  on  response  rate  to  mailed  questionnaires,  and 
concluded  that  the  number  of  subject  contacts  and  the 
importance  of  the  questionnaire  were  the  largest  determi¬ 
nants  to  response  rate  [5]. 

In  the  conduct  of  an  epidemiologic  health  study  that 
utilizes  self-reported  data,  the  investigators  should  be 
cognizant  that  response  bias  may  occur  if  invitees  choose 
to  participate  based  on  actual  or  perceived  differences  in 
health  status.  To  evaluate  the  presence  or  absence  of 
response  bias  contingent  upon  health  status,  the  investi¬ 
gators  must  have  access  to  health  information  on 
responders  and  nonresponders.  Since  health  care  usage  data 
are  largely  unavailable  for  nonresponders,  it  may  be  very 
difficult  for  US  investigators  to  assess  the  health  status  of 
these  individuals.  In  fact,  most  studies  of  nonresponder 
health  have  been  conducted  in  Europe  where  universal 
health  coverage  is  more  common  [1-4,  6-9].  Several  of 
these  studies  found  that  individuals  who  were  more  likely 
to  respond  to  health  surveys  were  those  who  frequently 
experienced  pain  and/or  utilized  health  care  [6,  8,  9], 
among  the  “worried  well”  [1],  reported  poorer  subjective 
health  and  less-healthy  lifestyle  habits  [3],  and  were  not 
receiving  a  disability  benefit  [2,  4]. 

Results  from  these  European  studies  prompted  investi¬ 
gation  into  whether  a  response  bias  due  to  health 
differences  prior  to  enrollment  exists  among  a  US  military 
population  in  the  Millennium  Cohort  Study.  Current  liter¬ 
ature  suggests  that  a  response  bias  due  to  differential  health 
status  between  responders  and  nonresponders  would  likely 
cause  more  individuals  with  poor  health  to  join  the  Cohort 
[6,  8,  9].  Should  this  participation  bias  occur  and  remain 
unknown,  it  may  have  significant  inferential  limitations  or 
even  lead  to  misleading  conclusions.  For  nonresponse  bias 
to  impact  study  findings,  the  respondents  and  nonrespon¬ 
dents  must  differ  on  reported  data  and  the  response  rate 
must  be  sufficiently  low  so  that  this  difference  can  have  an 
appreciable  effect  [10].  For  example,  with  a  low  response 
rate,  prevalence  estimates  derived  from  cross-sectional 
surveys  may  be  biased  if  these  two  conditions  exist  to  an 
appreciable  degree.  In  contrast,  when  conducting  a  longi¬ 
tudinal  study,  the  effect  of  response  bias  may  be  less 
intrusive  as  the  outcome  measure  is  often  assessed  at  fol¬ 
low-up  and  dependent  upon  baseline  health  status.  Because 
the  Millennium  Cohort  Study  is  a  longitudinal  prospective 
study,  nonresponse  bias  is  less  likely  to  affect  results  in 


comparison  to  studies  of  cross-sectional  design.  Yet,  it 
remains  prudent  for  researchers  to  investigate  where  pos¬ 
sible,  response  biases  that  may  result  from  self-selection 
and  over  or  under-representation  of  healthy  or  ill  partici¬ 
pants  in  a  study  population.  In  a  longitudinal  health  study 
such  as  this,  military  personnel  of  poorer  health  are  less 
likely  to  deploy  and  be  exposed  to  deployment  related 
exposures.  This  would  likely  bias  prevalence  estimates  and 
might  impact  longitudinal  studies,  limiting  the  utility  of  the 
Cohort. 

Because  all  inpatient  and  outpatient  encounters  for 
medical  care  under  the  US  Military  Health  System  are 
electronically  recorded  for  active-duty  service  members, 
this  study  represents  a  unique  opportunity  to  study  response 
bias  based  on  the  health  care  use  of  both  responders  and 
nonresponders.  Furthermore,  these  results  may  be  gener- 
alizable  to  the  other  populations.  For  example,  previous 
reports  have  assessed  the  similarities  in  the  distribution  of 
job  titles  among  members  of  the  US  civilian  workforce  and 
US  military.  Sulsky  has  compared  the  distribution  of  job 
titles  among  members  of  the  US  Army  who  were  serving 
on  active  duty  in  2001  to  information  published  in  the  2000 
US  Bureau  of  Fabor  Statistics,  Occupational  Employment 
Statistics  Survey.  With  the  exception  of  soldiers  working  in 
military- specific  jobs,  there  was  a  high  degree  of  correla¬ 
tion  in  the  types  of  job  titles.  While  the  percent  of  civilian 
and  military  workforce  in  transportation,  and  construction 
were  similar,  the  percent  of  military  working  in  other  jobs 
was  over-  or  under-represented  compared  to  their  civilian 
counterparts  [11].  In  summary,  it  is  important  to  under¬ 
stand  possible  biases  associated  with  the  Millennium 
Cohort  Study,  the  largest  prospective  study  to  ever  be 
conducted  with  a  military  population.  Additionally,  this 
study  may  provide  one  of  the  few  opportunities  to  accu¬ 
rately  assess  the  health  of  invitees  in  the  year  prior  to  study 
invitation  in  a  large  population-based  Cohort  of  US  mili¬ 
tary  personnel,  comparable  in  many  ways  to  civilian 
populations. 

Materials  and  methods 

Population  and  data  sources 

The  invited  Millennium  Cohort  Study  participants  were 
randomly  selected  from  all  US  military  personnel  serving 
in  October  2000,  oversampling  those  who  had  been  pre¬ 
viously  deployed,  Reserve  and  National  Guard  members, 
and  women,  to  ensure  sufficient  power  to  detect  differences 
in  smaller  subgroups  of  the  population.  The  probability- 
based  sample,  representing  ~  11.3%  of  the  2.2  million  men 
and  women  in  service  as  of  October  1 ,  2000,  was  provided 
by  the  Defense  Manpower  Data  Center  in  California. 
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World  Wide  Web  and  US  Postal  Service-based  enrollment 
began  in  July  2001,  and  a  modified  Dillman  approach  was 
utilized  to  contact  the  invited  service  members.  The 
enrollment  cycle  ended  on  June  30,  2003,  with  77,047 
consenting  participants.  The  methodology  of  the  Millen¬ 
nium  Cohort  Study  has  been  described  elsewhere  in  detail 
[12].  For  the  purposes  of  this  analysis,  only  those  invited 
personnel  on  active  duty  were  considered  ( n  =  140,842), 
since  comparable  health  care  utilization  records  for 
Reserve  and  National  Guard  personnel  may  not  exist.  The 
population  for  this  study  was  further  restricted  by  exclud¬ 
ing  those  active-duty  service  members  who  were  deployed 
to  a  combat  area  at  any  time  during  the  year  prior  to 
enrollment  or  during  the  enrollment  cycle  itself,  due  to  the 
likelihood  of  differential  access  to  care.  This  research  was 
conducted  in  compliance  with  all  applicable  federal  regu¬ 
lations  governing  the  protection  of  human  subjects  in 
research  (Protocol  NHRC. 2000. 007). 

Electronic  personnel  data  as  of  sample  construction  in 
October  2000  included  gender,  birth  date,  highest  educa¬ 
tion  level,  marital  status,  race/ethnicity,  service  branch 
(Army,  Navy,  Coast  Guard,  Air  Force,  and  Marine  Corps), 
primary  and  secondary  military  occupations,  and  a  personal 
identifier.  Additionally,  missing  electronic  data  on  age, 
education,  marital  status,  race/ethnicity,  and  occupation 
were  supplemented  with  available  electronic  personnel 
data  from  July  2000  to  January  2001.  This  process  of 
backfilling  missing  information  reduced  the  percentage  of 
individuals  missing  data  for  at  least  one  important  demo¬ 
graphic  characteristic  from  2.8%  to  1.1%  of  the  invited 
personnel,  resulting  in  an  analysis  population  of  68,103. 

Health  care  data 

To  assess  the  effect  of  prior  health  care  use  on  enrollment 
into  the  Millennium  Cohort,  the  frequency  and  types  of 
health  care  encounters  were  captured  using  Department  of 
Defense  electronic  hospitalization  and  outpatient  records 
during  the  period  from  July  1,  2000  to  June  30,  2001. 
Through  an  existing  data  use  agreement,  worldwide  elec¬ 
tronic  hospitalization  and  outpatient  records  for  all  US 
service  members  were  accessed  through  a  secure  portal. 
Through  this  service,  hospitalization  and  outpatient  records 
for  all  US  service  members  were  downloaded  to  a  secure 
server  and  then  matched  to  study  participants  by  personal 
identifier.  Because  healthcare  is  provided  at  no  cost  to  all 
US  service  members,  we  expect  to  non-differentially  cap¬ 
ture  all  hospitalizations  and  outpatient  visits  for  both 
responders  and  nonresponders  represented  in  this  study. 

Hospitalization  data  included  admission  date  and  up  to 
eight  discharge  diagnoses,  while  outpatient  data  included 
care  date  with  up  to  four  diagnoses.  Diagnoses  were  coded 


according  to  the  International  Classification  of  Diseases, 
Ninth  Revision,  Clinical  Modification  (ICD-9-CM)  [13]. 
Odds  of  response  to  the  Millennium  Cohort  questionnaire 
among  those  hospitalized  for  “any  cause”  were  examined, 
excluding  diagnoses  for  complications  of  pregnancy, 
childbirth,  and  puerperium.  In  addition,  each  of  the  15 
broad  ICD-9-CM  categories,  and  the  supplementary  cat¬ 
egories  for  factors  influencing  health  status  and  contact 
with  health  services,  were  investigated  separately  to  cal¬ 
culate  the  odds  of  response  to  the  questionnaire  among 
those  with  a  diagnosis  in  each  category.  Analysis  of 
individual  diagnoses  within  the  broad  diagnostic  catego¬ 
ries  of  hospitalizations  significantly  associated  with 
response  focused  on  the  five  most  frequent  three-digit 
diagnostic  codes  to  examine  their  association  with 
response.  Additionally,  specific  ICD-9-CM  diagnoses  that 
have  previously  been  indicative  of  chronic  multisymptom 
illness  were  explored  [14-25].  To  further  evaluate  health 
care  usage  differences  between  responders  and  nonre¬ 
sponders,  the  mean  number  of  days  spent  hospitalized  or 
in  outpatient  care  during  the  year  prior  to  enrollment  was 
also  compared. 

Statistical  analysis 

Descriptive  analyses  of  demographic  and  military  charac¬ 
teristics,  by  both  hospitalization  and  survey  response  status, 
were  completed.  Univariate  analyses,  including  tests, 
were  employed  to  assess  the  significance  of  unadjusted 
associations.  An  exploratory  analysis  was  conducted  to 
examine  regression  diagnostics,  significant  associations, 
and  possible  confounding,  while  simultaneously  adjusting 
for  other  variables.  Multivariable  logistic  regression  was 
used  to  compare  the  adjusted  odds  of  response  to  the 
Millennium  Cohort  questionnaire  among  those  with  and 
without  hospitalization  experience  for  any  cause  during  the 
year  prior  to  enrollment.  Individual  multivariable  logistic 
regression  models  [26]  were  constructed  for  each  diag¬ 
nostic  category,  as  previously  defined.  All  models  were 
adjusted  for  age,  gender,  race/ethnicity,  education,  marital 
status,  military  pay  grade,  military  branch  of  service,  and 
military  occupation. 

Lastly,  the  mean  number  of  days  that  responders  and 
nonresponders  utilized  inpatient,  outpatient,  and  both  types 
of  care  was  calculated  and  compared  by  response  status 
using  analysis  of  variance.  Of  the  3,833  individuals  who 
appeared  in  the  hospitalization  database,  207  did  not  have  a 
computable  length  of  stay,  so  the  mean  length  of  stay  for 
all  hospitalizations  (mean  =  3.499  days)  was  assigned. 
The  total  number  of  days  lost  to  care  over  the  year  was 
considered  to  be  total  days  hospitalized  plus  one  half-day 
lost  to  care  for  each  outpatient  visit. 
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All  analyses  were  completed  using  SAS  software  (ver¬ 
sion  9.1,  SAS  Institute,  Inc.,  Cary,  North  Carolina),  and 
adjusted  odds  ratios  (ORs),  95%  confidence  intervals  (CIs), 
and  adjusted  means  were  calculated  for  personnel  with 
complete  covariate  data  [27]. 

Results 

Descriptive  analyses  are  shown  in  Table  1.  Those  more 
likely  to  be  hospitalized  were  women,  those  younger  than 
24  or  older  than  45  years  of  age,  those  with  a  high  school 
diploma  or  less,  enlisted  personnel,  those  not  married, 
members  of  the  Army,  and  participants  in  various  occu¬ 
pations,  most  notably  health  care  workers  (p  <  0.05). 
Several  demographic  factors  were  also  associated  with 
survey  response  (Table  2).  There  was  no  association, 
however,  between  any-cause  hospitalization  and  response 
(adjusted  OR  =  0.93,  95%  Cl:  0.85,  1.02)  after  adjusting 
for  gender,  age,  race/ethnicity,  educational  level,  military 
rank,  marital  status,  service  branch,  and  occupation. 

Millennium  Cohort  responders  and  nonresponders  were 
also  evaluated  for  the  time  spent  accessing  inpatient  or 
outpatient  care  in  the  year  prior  to  enrollment.  The  mean 
and  median  number  of  days  spent  in  aggregated  inpatient 
or  outpatient  care  differed  by  less  than  a  half-day  and  was 
statistically  significant  at  the  a  =  0.05  level  (Table  3). 

In  separate  multivariable  logistic  regression  analyses, 
adjusted  odds  of  response  were  calculated  for  16  broad 
categories  of  ICD-9-CM  coded  diagnoses.  There  was  no 
association  between  response  and  hospitalizations,  except 
that  responders  were  at  significantly  lower  adjusted  odds 
for  mental  disorder  hospitalizations  (Fig.  1).  When  outpa¬ 
tient  diagnoses  were  examined,  responders  were  less  likely 
to  have  outpatient  visits  for  mental  disorders  (OR  =  0.88, 
95%  Cl:  0.83,  0.93).  In  addition,  responders  were  signifi¬ 
cantly  more  likely  to  have  outpatient  visits  for  neoplasms; 
diseases  of  the  nervous,  circulatory,  musculoskeletal,  and 
respiratory  systems;  diseases  of  the  skin  and  subcutaneous 
tissues;  endocrine,  nutritional,  and  metabolic  diseases;  and 
for  supplementary  (V)  codes,  after  adjusting  for  all  vari¬ 
ables  in  the  models,  although  these  adjusted  ORs  were  all 
less  than  1.2  (Fig.  2). 

Those  ICD-9-CM  categories  with  statistically  significant 
results  were  further  evaluated  to  identify  the  top-five  most 
frequent  diagnoses.  The  five  most  common  inpatient 
mental  disorder  diagnoses  included  nondependent  abuse  of 
drugs,  adjustment  reactions,  affective  psychoses,  alcohol 
dependence  syndrome,  and  personality  disorders;  those 
hospitalized  for  adjustment  reactions  and  personality  dis¬ 
orders  were  significantly  less  likely  to  respond  (Table  4). 
Because  diseases  such  as  asthma,  fibromyalgia,  and  chronic 
fatigue  syndrome  were  reported  to  be  more  prevalent  in 


1990-1991  Gulf  War  veterans  than  nondeploy ed  veterans 
of  the  same  era,  we  sought  to  determine  if  there  was  a 
response  bias  based  on  these  medical  conditions.  After 
adjusting  for  gender,  age,  education,  marital  status,  race/ 
ethnicity,  pay  grade,  branch  of  service,  and  occupation, 
there  was  no  association  between  response  status  and 
asthma,  fibromyalgia,  or  chronic  fatigue  syndrome  as 
measured  by  either  hospitalizations  or  outpatient  visits 
(data  not  shown). 

Discussion 

In  this  study,  we  found  no  substantial  health  differences 
between  Millennium  Cohort  responders  and  nonresponders 
when  measured  by  health  care  use  in  the  12  months  pre¬ 
ceding  study  invitation.  With  the  exception  of  mental 
disorders,  there  were  no  differences  in  response  based  on 
prior  hospitalization  experiences  between  responders  and 
nonresponders.  Responders  were  slightly  more  likely  to 
have  utilized  outpatient  health  care  across  eight  broad  ICD- 
9-CM  categories,  although  the  small  ORs  observed  were 
close  to  unity,  implying  little  to  no  clinically  meaningful 
differences.  Additionally,  findings  demonstrated  that 
responders  and  nonresponders  were  very  similar  in  terms  of 
the  number  of  days  spent  utilizing  inpatient  and  outpatient 
health  care  services. 

Responders  and  nonresponders  appeared  to  be  of  com¬ 
parable  health  based  on  inpatient  data.  This  finding  likely 
provides  the  best  assessment  that  responders  and  nonre¬ 
sponders  did  not  have  significant  health  differences  in  the 
year  prior  to  invitation  for  enrollment.  Becoming  hospi¬ 
talized  generally  involves  an  objective  decision  by  a 
medical  provider  indicating  the  patient’s  condition  war¬ 
rants  hospitalization.  In  contrast,  the  decision  to  seek 
outpatient  care  may  be  a  personal  decision  to  seek  con¬ 
sultation,  preventive  medicine,  or  medical  treatment 
services. 

In  further  analyses  of  outpatient  visits  (data  not  shown), 
we  observed  that  responders  were  more  likely  to  have  an 
outpatient  diagnosis  for  unspecified  neoplasms,  disorders 
of  lipoid  metabolism,  diseases  of  capillaries,  contact  der¬ 
matitis  and  other  eczema,  other  and  unspecified  disorders 
of  joints,  and  acute  pharyngitis,  and  allergic  rhinitis,  all  of 
which  appear  to  indicate  that  responders  may  have  a  higher 
disease  burden  than  nonresponders.  However,  responders 
were  also  more  likely  to  have  sought  medical  consultation 
without  complaint  or  sickness,  to  have  completed  follow¬ 
up  examinations,  or  had  a  visit  to  correct  a  disorder  of 
vision  refraction  and  accommodation,  indicating  that 
responders  also  appear  to  utilize  health  care  for  reasons 
other  than  disease  or  injury  treatment.  Additionally,  indi¬ 
vidual  decisions  to  seek  outpatient  health  care  may  be 
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Table  1  Characteristics 
associated  with  hospitalization 
in  the  year  prior  to  invitation 
among  active-duty  members 
invited  to  participate  in  the 
Millennium  Cohort  in  200 la 


a  Selected  based  on  the 
following  criteria:  must  have 
served  on  continuous  active- 
duty  throughout  health  care 
observation  period  and  not 
deployed  at  any  time  during  the 
health  care  observation  period 
or  study  enrollment  period 

b  Characteristics  defined  as  of 
October  2000  when  the 
potential  participant  pool  was 
identified.  All  y2  tests  of 
significance,  except  race/ 
ethnicity,  were  statistically 
significant  at  P  <  0.05 


Characteristics*5 

Total 

Hospitalized 

Not  hospitalized 

(n  =  68,103) 

(n  =  2,335) 

(n  =  65,768) 

n  (%) 

n  (%) 

n  (%) 

Gender 


Male 

50,447  (74.1) 

1,543  (66.1) 

48,904  (74.4) 

Female 

17,656  (25.9) 

792  (33.9) 

16,864  (25.6) 

Age  (years) 

17-24 

19,054  (28.0) 

714  (30.6) 

18,340  (27.9) 

25-34 

17,709  (26.0) 

563  (24.1) 

17,146  (26.1) 

35-44 

18,101  (26.6) 

543  (23.3) 

17,558  (26.7) 

>45 

13,239  (19.4) 

515  (22.1) 

12,724  (19.4) 

Race/ethnicity 

White,  non-Hispanic 

41,471  (60.9) 

1,404  (60.1) 

40,067  (60.9) 

Black,  non-Hispanic 

15,185  (22.3) 

563  (24.1) 

14,622  (22.2) 

Other 

11,447  (16.8) 

368  (15.8) 

11,079  (16.9) 

Highest  educational  level 

High  school  diploma  or  less 

40,433  (59.4) 

1,560  (66.8) 

38,873  (59.1) 

Some  college 

18,587  (27.3) 

530  (22.7) 

18,057  (27.5) 

Bachelor’s  degree 

4,976  (7.3) 

125  (5.4) 

4,851  (7.4) 

Graduate  school 

4,107  (6.0) 

120  (5.1) 

3,987  (6.1) 

Rank 

Enlisted 

60,828  (89.3) 

2,160  (92.5) 

58,668  (89.2) 

Officer 

7,275  (10.7) 

175  (7.5) 

7,100  (10.8) 

Marital  status 

Single 

25,019  (36.7) 

888  (38.0) 

24,131  (36.7) 

Married 

39,826  (58.5) 

1,313  (56.2) 

38,513  (58.6) 

Other 

3,258  (4.8) 

134  (5.7) 

3,124  (4.8) 

Service  branch 

Army 

22,909  (33.6) 

1,017  (43.6) 

21,892  (33.3) 

Air  Force 

20,483  (30.1) 

576  (24.7) 

19,907  (30.3) 

Navy  and  Coast  Guard 

18,000  (26.4) 

554  (23.7) 

17,446  (26.5) 

Marine  Corps 

6,711  (9.9) 

188  (8.1) 

6,253  (9.9) 

Occupational  codes 

Combat  specialists 

12,522  (18.4) 

442  (18.9) 

12,080  (18.4) 

Functional  support  and  admin 

14,567  (21.4) 

509  (21.8) 

14,058  (21.4) 

Electrical/mechanical  equip. 

11,442  (16.8) 

373  (16.0) 

11,069  (16.8) 

repair 

Electronic  equipment  repair 

6,383  (9.4) 

179  (7.7) 

6,204  (9.4) 

Health  care 

6,228  (9.1) 

267  (11.4) 

5,961  (9.1) 

Communications/intelligence 

5,627  (8.3) 

168  (7.2) 

5,459  (8.3) 

Service  and  supply 

5,567  (8.2) 

217  (9.3) 

5,350  (8.1) 

Craft  workers 

2,102  (3.1) 

64  (2.7) 

2,038  (3.1) 

Other  technical  and  allied 

1,862  (2.7) 

50  (2.1) 

1,812  (2.8) 

Students,  prisoners,  and  other 

1,803  (2.7) 

66  (2.8) 

1,737  (2.6) 

influenced  by  accessibility,  perceived  condition  severity, 
concerns  over  social  stigmatization  associated  with  health- 
related  personal  habits,  or  illness  conditions.  Furthermore, 
military  members  who  receive  flight  and  other  specialty 
pay  jeopardize  these  earnings  should  they  be  diagnosed  with 
certain  duty-limiting  medical  conditions.  All  of  these  possi¬ 
bilities  make  the  use  of  outpatient  care  a  more 


ambiguous  measure  of  health  in  comparison  with  inpatient 
health  care  use. 

Invitees  with  inpatient  or  outpatient  visits  for  mental 
disorders  were  significantly  less  likely  to  participate  in  the 
study.  Upon  closer  examination  of  these  data,  nondepen¬ 
dent  abuse  of  drugs  and  adjustment  reactions  were  the  two 
most  frequent  three-digit  ICD-9-CM  diagnoses  in  both  the 
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Table  2  Characteristics 
associated  with  response  to 
invitation  to  enroll  in  the 
Millennium  Cohort  among 
active-duty  members  invited  in 
2001 


a  Selected  based  on  the 
following  criteria:  must  have 
served  on  continuous  active 
duty  throughout  health  care 
observation  period  and  not 
deployed  at  any  time  during  the 
health  care  observation  period 
or  study  enrollment  period. 
Characteristics  defined  as  of 
October  2000  when  the 
potential  participant  pool  was 
identified 

b  ORs  and  associated  95%  CIs 
from  multiple  logistic 
regression  were  adjusted  for 
gender,  age,  education,  marital 
status,  race/ethnicity,  pay  grade, 
branch  of  service  and 
occupation.  CIs  that  exclude 
1.00  were  significant  at  the 
P  <  0.05  level 


Characteristicsa 

Non-responders 
(ti  =  47,036) 
n  (%) 

Responders 
(n  =  21,067) 
n  (%) 

ORb  95%  CIb 

Hospitalization  in  year  prior  to  invitation 

Not  hospitalized 

45,403  (96.5) 

20,365  (96.7) 

1.00 

Hospitalized 

1,633  (3.5) 

702  (3.3) 

0.93  0.84,  1.03 

Gender 

Male 

35,304  (75.1) 

15,143  (71.9) 

1.00 

Female 

11,732  (24.9) 

5,924  (28.1) 

1.43  1.37,  1.49 

Age  (years) 

17-24 

15,520  (33.0) 

3,534  (16.8) 

1.00 

25-34 

12,988  (27.6) 

4,721  (22.4) 

1.47  1.39,  1.55 

35-44 

11,067  (23.5) 

7,034  (33.4) 

2.45  2.31,  2.59 

>45 

7,461  (15.9) 

5,778  (27.4) 

2.71  2.54,  2.89 

Race/ethnicity 

White,  non-Hispanic 

27,734  (59.0) 

13,737  (65.2) 

1.00 

Black,  non-Hispanic 

11,568  (24.6) 

3,617  (17.2) 

0.56  0.53,0.58 

Other 

7,734  (16.4) 

3,713  (17.6) 

0.81  0.77,  0.85 

Highest  educational  level 

High  school  diploma  or  less 

29,459  (62.6) 

10,974  (52.1) 

1.00 

Some  college 

12,745  (27.1) 

5,842  (27.7) 

1.15  1.07,  1.23 

Bachelor’s  degree 

2,842  (6.0) 

2,134  (10.1) 

1.16  1.07,  1.27 

Graduate  school 

1,990  (4.2) 

2,117  (10.1) 

1.20  1.06,  1.35 

Rank 

Enlisted 

43,328  (92.1) 

17,500  (83.1) 

1.00 

Officer 

3,708  (7.9) 

3,567  (16.9) 

1.45  1.32,  1.60 

Marital  status 

Single 

19,536  (41.5) 

5,483  (26.0) 

1.00 

Married 

25,428  (54.1) 

14,398  (68.3) 

1.28  1.23,  1.34 

Other 

2,072  (4.4) 

1,186  (5.6) 

1.15  1.05,  1.25 

Service  branch 

Army 

15,115  (32.1) 

7,794  (37.0) 

1.00 

Air  Force 

14,060  (29.9) 

6,423  (30.5) 

0.60  0.56,0.64 

Navy  and  Coast  Guard 

12,770  (27.2) 

5,230  (24.8) 

0.64  0.61,0.67 

Marine  Corps 

5,091  (10.8) 

1,620  (7.7) 

0.67  0.63,  0.72 

Occupational  codes 

Combat  specialists 

8,915  (19.0) 

3,607  (17.1) 

1.00 

Functional  support  and  admin 

9,887  (21.0) 

4,680  (22.2) 

1.21  1.14,  1.28 

Electrical/mechanical  equip. 

8,241  (17.5) 

3,201  (15.2) 

1.22  1.15,  1.30 

repair 

Electronic  equipment  repair 

4,193  (8.9) 

2,190  (10.4) 

1.41  1.31,  1.51 

Health  care 

3,954  (8.4) 

2,274  (10.8) 

1.32  1.23,  1.41 

Communic  ation  s/ intelligence 

3,791  (8.1) 

1,836  (8.7) 

1.32  1.23,  1.42 

Service  and  supply 

3,947  (8.4) 

1,620  (7.7) 

1.09  1.02,  1.18 

Craft  workers 

1,540  (3.3) 

562  (2.7) 

1.13  1.01,  1.26 

Other  technical  and  allied 

1,250  (2.7) 

612  (2.9) 

1.30  1.17,  1.46 

Students,  prisoners,  and  other 

1,318  (2.8) 

485  (2.3) 

1.22  1.08,  1.37 

inpatient  and  outpatient  setting.  Responders  were  signifi¬ 
cantly  less  likely  to  have  inpatient  or  outpatient  diagnoses 
of  adjustment  reactions,  and  significantly  less  likely  to  have 


outpatient  visits  for  nondependent  abuse  of  drugs.  Finding 
less  participation  by  those  with  mental  disorders  is  not 
widely  reported.  In  an  analysis  of  nonparticipants  for  the 
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Table  3  Number  of  days  spent  in  inpatient  and  outpatient  care  in  the 
year  prior  to  invitation  among  active-duty  Millennium  Cohort  survey 
responders  and  nonresponders 


Data  source 

Meana 

Median,  IQRb 

P-valuec 

Inpatient  hospitalization 

Responder 

0.21 

0.00,  0.00 

0.034 

Nonresponder 

0.25 

0.00,  0.00 

Outpatient  encounters 

Responder 

4.94 

3.00,  5.50 

<0.001 

Nonresponder 

4.58 

2.50,  5.00 

Combined 

Responder 

5.15 

3.00,  5.50 

<0.001 

Nonresponder 

4.83 

3.00,  5.00 

a  Mean  number  of  days  was  calculated  for  the  year  prior  to  Millen¬ 
nium  Cohort  Study  enrollment,  July  1,  2000,  to  June  30,  2001 

b  Interquartile  range  indicates  the  difference  between  the  third 
quartile  and  the  first  quartile 

c  P- value  significance  was  calculated  by  analysis  of  variance 

Canadian  Study  of  Health  and  Aging,  the  authors  found  that 
cognitively  impaired  subjects  had  higher  refusal  rates  [28]. 
It  is  also  possible  that  individuals  diagnosed  with  nonde¬ 
pendent  abuse  of  drugs  and  other  mental  disorders  feel  a 
certain  degree  of  social  stigmatization  and  prefer  not  to 
share  their  health  information  with  strangers.  For  example, 
Hoge  et  al.  documented  that  some  military  members  are 
likely  to  avoid  seeking  health  care  for  mental  disorders  due 
to  perceived  concerns  over  the  social  stigmatization  and  the 
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Fig.  1  Adjusted  ORs  and  95%  CIs  for  odds  of  response  to  the 
Millennium  Cohort  questionnaire,  adjusted  for  gender,  age,  education, 
marital  status,  race/ethnicity,  pay  grade,  branch  of  service  and 
occupation  for  responders  and  nonresponders  who  were  hospitalized 
from  July  1,  2000,  to  June  30,  2001,  1  year  prior  to  the  start  of  study 
enrollment 


Fig.  2  Adjusted  ORs  and  95%  CIs  for  odds  of  response  to  the 
Millennium  Cohort  questionnaire,  adjusted  for  gender,  age,  education, 
marital  status,  race/ethnicity,  pay  grade,  branch  of  service  and 
occupation  for  responders  and  nonresponders  who  utilized  outpatient 
care  from  July  1,  2000,  to  June  30,  2001,  1  year  prior  to  the  start  of 
study  enrollment 

possible  adverse  impact  mental  disorder  diagnoses  may 
have  on  careers  [29,  30].  It  is  also  plausible  that  those  with 
adjustment  reactions  felt  overwhelmed  at  the  time  of  invi¬ 
tation  and  felt  that  they  were  unable  to  add  additional 
responsibilities,  such  as  committing  to  a  21 -year  follow-up, 
thus  making  them  less  likely  to  respond. 

These  findings  must  be  interpreted  within  the  limitations 
and  strengths  of  the  study.  One  potential  limitation  is  that 
this  study  was  restricted  to  those  Millennium  Cohort 
members  whose  health  care  data  were  available  during  the 
entire  observation  period,  which  meant  removing  service 
members  in  the  Reserve  or  Guard  and  those  who  had 
deployed  during  the  observation  period  or  during  study 
enrollment.  This  restriction  was  implemented  to  ensure 
responders  and  nonresponders  had  equal  opportunity  to 
access  military  health  care.  Because  healthier  service 
members  may  be  selected  to  deploy,  this  restriction  may 
have  made  this  sample  less  representative  of  the  entire 
Millennium  Cohort,  or  the  US  military  in  general.  How¬ 
ever,  we  have  no  reason  to  believe  that  this  restriction 
would  have  differentially  selected  responders  or  nonre¬ 
sponders  with  poorer  health  for  these  analyses. 

In  contrast  to  potential  limitations,  this  study  should  be 
considered  robust  by  several  measures.  First,  it  was  based 
on  21,067  responders  and  47,036  nonresponders,  repre¬ 
senting  one  of  the  largest  studies  ever  of  nonresponse  bias  to 
enrollment  invitation.  Second,  objective  hospitalization  and 
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Table  4  Frequencies  of  the  five  most  common  inpatient  diagnoses  among  statistically  significant  broad  ICD-9-CM  diagnostic  categories  for 
Millennium  Cohort  Study  responders  and  nonresponders  with  prior  year  hospitalizationsa 


ICD-9-CM  category13 

Diagnoses 

Frequency  of  diagnoses 

Odds  ratiod 

95%  CId 

Responders  (n  =  21,067) 

Non-responders  (n  =  47,036)c 

n 

(%> 

n 

(%) 

Mental  disorders  (codes  290-319) 

305 

Nondependent  abuse  of  drugs 

33 

0.2 

109 

0.2 

0.74 

0.49,  1.10 

309 

Adjustment  reaction 

24 

0.1 

117 

0.2 

0.57 

0.36,  0.90 

296 

Affective  psychoses 

30 

0.1 

80 

0.2 

0.96 

0.62,  1.49 

303 

Alcohol  dependence  syndrome 

14 

0.1 

55 

0.1 

0.59 

0.32,  1.09 

301 

Personality  disorders 

4 

0.0 

52 

0.1 

0.26 

0.09,  0.74 

a  Hospitalizations  were  observed  from  July  1,  2000,  to  June  30,  2001,  1  year  prior  to  the  start  of  Millennium  Cohort  Study  enrollment 
b  ICD-9-CM,  International  Classification  of  Diseases,  Ninth  Revision,  Clinical  Modification 
c  The  reference  group  for  the  multiple  logistic  regression  models  is  nonresponders 

d  ORs  and  associated  95%  CIs  from  multiple  logistic  regression  were  adjusted  for  gender,  age,  education,  marital  status,  race/ethnicity,  pay 
grade,  branch  of  service  and  occupation.  Measures  in  bold  were  statistically  significant  at  the  P  <  0.05  level 


outpatient  medical  data  were  utilized  to  investigate  diag¬ 
nostic  categories  associated  with  enrollment  as  well  as  to 
compute  measures  of  days  utilizing  hospitalization  and 
outpatient  care,  allowing  for  a  thorough  analysis  of  this 
subject.  Finally,  because  military  health  care  is  equally 
accessible,  irrespective  of  gender,  age,  or  pay  grade,  all 
responders  and  nonresponders  should  have  had  equal  access 
to  health  care  during  the  observation  period,  reducing  the 
potential  for  bias  dependent  on  socioeconomic  differences. 

In  conclusion,  these  analyses  illustrated  that  responders 
and  nonresponders  were  comparable  with  regard  to  health 
at  the  onset  of  the  Millennium  Cohort  Study,  as  measured 
by  hospitalization  and  days  spent  accessing  health  care 
services.  No  substantial  or  clinically  relevant  health-related 
response  bias  was  found  among  the  participants,  providing 
reassurance  that  individuals  in  the  Millennium  Cohort 
exhibit  comparable  health  attributes  that  are  representative 
of  the  target  population  for  this  study. 
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